A Multi-modal Auditory-Visual-Tactile e-Learning Framework

With a high number of countries closing learning institutions due to the restrictions in response to the COVID-19 pandemic, over 80% of the world’s students was not attending school. As a response to this challenge, many educational institutions are increasing their efforts to utilise various educational technologies and provide remote learning opportunities. One of the biggest drawbacks of the majority of these existing solutions is limited support for hands-on laboratory work and practical experiences. This is especially relevant to science, technology, engineering, and mathematics (STEM) departments, which must continuously develop their laboratories and pedagogical tools to provide their students with effective study plans. To facilitate a safe, digital access to laboratories, a novel haptic-enabled framework for hands-on e-Learning is introduced in this work. The framework enables a fully-immersive tactile, auditory, and visual experience. This is achieved by combining virtual reality (VR) tools, with a novel wearable haptic device, which is designed by augmenting a low-cost commercial off-the-shelf (COTS) controller with vibrotactile actuators. For this purpose, the Unity game engine and the Valve Knuckles EV3 controllers are adopted. To demonstrate the potential of the proposed framework, a human subject study is presented. Results suggest that the proposed haptic-enabled framework improves the student engagement and illusion of presence. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.